1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and a program that are suitable for use for suppressing noise.
2. Description of the Related Art
In recent years, there has been a growing trend toward smaller pixels for image sensors, which has entailed a tendency toward increased noises in image signals obtained from an image sensor. As a solution to this, there are known approaches through signal processing, in which multirate signal processing is used to suppress the noises.
Japanese Patent Application Laid-Open No. 2008-15741, for example, discloses a technique in which an image signal is divided into a plurality of frequency components, and noises are suppressed in each divided image before recombining. For example, an image signal is divided for each frequency band as illustrated in FIG. 11, and noise suppression processing is performed on each divided image. Various dividing methods are available to obtain the frequency components. To generate a low frequency image, for example, a prefilter is applied to an original image to thin its size to a half. To generate a high frequency image, a low frequency image is enlarged such that its size is similar to that of the original image, and then the image subjected to the enlargement processing is subtracted from the original image.
Through the division into the frequency components as described above, the noise suppression can be set by an application amount suitable for the frequency component of each divided image. In addition, the reduction in size of each image allows the noise reduction with a short filter length. Note that high frequency noises are inconspicuous visually due to a finer granularity of the noise. Conversely, low frequency noises are conspicuous visually due to a rougher granularity. Hence, it is desirable that the low frequency noises are removed by a larger amount than the high frequency noises.
In addition, it is possible to perform processing of combining divided images, which have been divided into the frequency components, with a simple calculation of adding an enlarged low frequency image to a high frequency image. As it is apparent from the calculation method for the combining processing, though, when a divided image suffers a blur due to the noise suppression processing, an image obtained after the combining processing also suffers a blur remaining in a corresponding frequency band. The method of noise suppression through the division into frequency bands is problematic in that, even though this method allows noise suppression by a different amount for each frequency band, the effect of the noise suppression should be restrained such that a blur is tolerable in each frequency band.
In light of this, another noise suppression technique is disclosed in Japanese Patent Application Laid-Open No. 2009-199104. In this technique, a plurality of reduced images is generated from an image signal, and, then, the noise suppression processing is performed on the plurality of reduced images. Also, an edge signal is extracted to obtain, for each image, a combination ratio based on the edge signal. The plurality of image signals is then combined on the basis of the combination ratio.
As illustrated in FIG. 12, for example, an image reduced to a ½ size and an image reduced to a ¼ size in both a horizontal direction and a vertical direction are generated. The noise suppression processing is then performed on each image, and an edge signal is extracted. To combine the images, a combination ratio is decided on the basis of the edge signal, such that a larger sized image is used for a portion with a strong edge signal and an image, which has been reduced and then enlarged to an original size, is used for a portion with a weak edge signal.
For edge portions of an image obtained after the combining, the arrangement as described above allows a greater use of original-sized images, while not using reduced images as much. Hence, even when an application of the noise suppression processing to a reduced image results in a blur of an edge portion, the blur does not affect the last image. This allows a stronger application of the noise suppression to a reduced image, which enables an image to be obtained with an increased effect of the noise suppression in low frequency components without causing a blur in the image obtained after the combining.
The combination ratio, however, is calculated from an edge detected in an image. Hence, as illustrated in FIG. 13A, for example, noises that are overlaid on the image may obscure an edge, thereby interfering with the combination ratio. In other words, when the combination ratio varies in a continuous edge, a high resolution image is used for a portion with a high combination ratio, while a low resolution image is used for a portion with a low combination ratio. This may result in a problem that the edge in an image obtained after the combining may appear discontinuous as illustrated in FIG. 13B or the edge may vary in thickness in an unstable manner, leading to a degradation in image quality. Note that the image quality degradation occurred during the combining has a propensity for visual conspicuousness if it is caused in a portion with a relatively high resolution.
As described above, the technique of dividing an image per frequency and suppressing noises in each image presents a problem that it is difficult to produce a full effect of the noise suppression. The technique of generating a plurality of reduced images to suppress noises in each image and combining the images using edge signals presents a problem that the edge signals are affected by the noises, resulting in obscureness in a composite image.